Time Machine does not see old Contacts

I use El Capitan 10.11.6 with two external hard drives for back up. When I launch Apple Contacts and go to Time Machine intent to restore an older contact, going back a few months, I will hear my external drives spin but the Contact app has a blank "no cards" screen where my lists of contacts should be.

However sometimes it works as it should. I've tried cold starts with nothing else running. I've tried getting the drives to spin up before I launch the contact app. I can see any rhyme or reason for this behavior.

I can restore other files using Time Machine so I think it works OK otherwise. Any suggestions for getting this to work consistently? Thanks